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Abstract 


The research compared and assessed 6" grade student large-scale assessment item indicators 
according to the revised competency-based Science curriculum in Latvia, in order to understand 
what improvements are needed in the large-scale assessment for aligning the national assessments 
with the new national curriculum, and thus ensuring successful implementation of the educational 
reform. The theoretical framework of the research was developed by using the frameworks of 
the Programme for International Student Assessment (PISA), and cognitive level was measured 
according to the Structure of Observed Learning Outcomes (SOLO) taxonomy. 
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Introduction 


Similarly as other countries around the world, Latvia undergoes a nation-wide 
curriculum reform in general education with an aim to develop 21" century skills. The 
reform focuses on the development of learners’ competencies — the ability to apply 
knowledge, skills, attitudes and values to various contexts, complex and changing real 
life situations (OECD, 2016). Scientific literacy is one of the key competencies for the 
21“ century learner (OECD, 2018), and it is defined as one of the learning goals that 
should be achieved at the end of the compulsory education in Latvia -Grade 9 (Regulation 
No.747, 2018). 

To support successful implementation of the reform, the alignment of the 
curriculum with pedagogy and assessment plays a crucial role (Fullan & Quinn, 2015). 
Previous research (France et al., 2017; Pestovs & Namsone, 2018) indicated that there 
is a difference between the learning content defined in the national curriculum and the 
learning content that is assessed in the national tests. Thus, the analysis of national tests 
and the degree to which the assessment is aligned to the new curriculum characterises 
success regarding the accomplishment of the reform aims. 

The purpose of the research was to review and evaluate 6" Grade national level 
tests in Science subject for two consecutive years — 2018 and 2019 in the context of the 
new curriculum. The main aim of the analysis was to gain an insight into the conditions 
significant for successful reform implementation. 

The theoretical framework with regard to the definition of scientific literacy was 
based on the OECD’s (Organisation for Economic Co-operation and Development) PISA 
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(Programme for International Student Assessment) Science assessment framework, and 
the definition of the item cognitive demand — on the SOLO (Structure of Observed 
Learning Outcomes) taxonomy (Biggs & Collis, 1982). 

To achieve the purpose, the following research questions were identified: 
1. What are the assessment indicators in 6" Grade national tests in Science in 2 years’ 
period in Latvia? 
2. Are there any changes between the years? 
3. To what extent the aspects measured in the national tests align with the revised 
curriculum? 


Research Methodology 


National level tests of years 2018 and 2019 were analysed by specially 
trained experts, individually and in focus group discussions. Firstly, the 
assessment indicators were determined individually, and then during several (at 
least three) iterations in a group setting the assessment indicators were revised 
and the cognitive level of each item according to the SOLO was defined. 
Additionally, for several items student answers were analysed to understand 
what answers were accepted during the marking, as national level tests were 
administrated and marked internally in schools. 

The assessment indicators of national level tests in Science were mapped 
according to the (a) scientific literacy categories (that represent the new 
curriculum) and (b) SOLO cognitive levels (I, II, I, IV). 

The limitations of the research methodology were that there was an evidence of 
inconsistency between the item cognitive demand and the school interim marking, which 
in fact reduces the cognitive demand of the item. For some items experts could not define 
the assessment indicator and these items were excluded from the research. 


Research Results 


As Table | demonstrates, most assessment indicators in Science national level 
test in 2019 were identified at SOLO II and I levels, while only 3 items (from 23) were 
identified at SOLO III level. Assessment indicators for items 19 and 22 were not defined, 
and items were omitted from the analysis, because of the inconsistency in decisions 
about assessment indicators. By comparison, from all 30 assessment indicators in 
Science national level test in 2018 only | item was identified at SOLO ITI level, the rest 
- SOLO II and I levels (see Table 2). 
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Table 1. Item assessment indicators of 6th grade national level test in 
Science, Year 2019. 


“nay Assessment Indicators eee 
1, Calculates the time from minutes to seconds | 
2. Deduces the Moon phase by modelling Il 
3. Knows why the day and night shifts | 
4. Knows how to verify that there is air in the room Ill 
5. Determines scale according to the stated conditions | 
6. Knows the necessary equipment for filtering ll 
7. Recognizes the mixture, which can be filtered | 
8. Recognizes the condition for the faster evaporation | 
ol Knows what is dissolution | 
10. Knows how to speed up the dissolution process Il 
11. Compares the numbers to determine the biggest ratio solute in the solution Il 
12. Recognizes the described observation Il 
13. Recognizes common used material properties Il 
14. Determines the distance by extrapolating the linear graphic Il 


15. Deduces the length from the graphic | 


16. Recognizes a working electric circuit Ill 

17. Explains the phenomenon of using observation and gas volume dependence of | 
temperature 

18. Knows renewable energy sources Il 

20. Recognizes plant by indications Il 

21. Recognizes parts of plants shown in the figure Il 

93. Predicts in a particular example the three elements of ecosystem dependency and the i 
consequences of one element of change 

24. Compares values to make the reliable conclusion Il 

25. Interprets complex graphical information Ill 
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Table 2. Item assessment indicators of 6th grade national level test in 
Science, Year 2018. 


iter Assessment Indicators eee 
No. levels 
1: Determines the lowest value by comparing the complex by analysing the information | 


Recognizes the relation between the concepts of "density" and "mass" and "volume" | 


Find the largest number in the table | 


2 
3 
4. Evaluate the way the data is displayed according to the obtained data Il 
5 


Determines the highest value by comparing the complex by analysing the information | 


Reads the values from the bar chart and the value from the graph to complex graphical 


e information. 
if Know that the type of precipitation depends on the temperature. Il 
8. Interprets graphical information in a specific (biology) context, evaluating the relevance of i 
statements to interpretation 
9: Reads the volume of liquid using the drawing of a measuring cylinder Il 
10. | Recognizes supportive, nervous, circulatory and respiratory systems by image | 
11. | Knows the oceans and seas are saltwater Il 
12. | Knows from which side the sunrise takes place in the northern hemisphere of the Earth | 
13. | Reads the largest and smallest sector from the chart Ill 
14. | Interprets bar chart data in context (astronomy) to form a verbal relationship | 
15. | Knows the Moon Phase succession and appearance | 
16. | Recognizes the smallest value from the bar chart Il 
17. | Knows the biggest causes of pollution | 
18. | Knows how to conserve natural resources | 
19. | Knows an example of action that leads to a reduction in air pollution | 
20. | Knows that sunglasses protect your eyes from light and ultraviolet radiation | 
21. | Knows renewables and non-renewable energy sources | 
22. | Knows the change of pole length depending on the time of day | 
23. | Transforms the data in a graphical way | 
24. | Finds and reads information Il 
25. | Knows that the experiment should be repeated three times to calculate the mean value Il 
26. | Knows that the dependent variable should be changed Il 
27. | Finds and reads information Il 
28. | Finds and reads literally found different information Il 
29. | Determines the appropriate graphical representation according to the scale. Il 
30. | Set higher values by analysing complex graphical information Il 


| 


Proceedings of the 3" International Baltic Symposium on Science and Technology Education, BalticSTE2019 | 


166 


Table 3. Alignment between item assessment indicators of 6th grade 
national level tests in Science and the revised curriculum 


framework. 
SOLO level 
| II Ill IV 
Competence 
Year | Year Year Year Year Year | Year | Year 
2018 | 2019 2018 2019 2018 | 2019 | 2018 | 2019 
Explain phenom- Pe 18, 0 
ule 10, 9, 17 
Scientific | © scientifically 8 4 DS, P23) 
literacy 
Interpret data 3, 14, oe * || Zl, 20, 
and evidence 22, 1 ene 14,11, | 16 ZnO) 
scientificall 23 2g 15 
y 29, 30 
Evaluate and 
: ae |. U9), 23, 
design scientific 5.6 6 6 13, 24 
enquiry 


Assessment indicators of 2018 and 2019 Science national level tests demonstrated 
inconsistency in two-year period both in scientific literacy competencies and SOLO 
levels. Most indicators matched with “interpreting data and evidence scientifically”. 
According to the SOLO levels, during two-year period only 4 items were identified at 
SOLO III level, and there are no items identified at SOLO IV level, which is critically 
important for the knowledge transfer and conceptual understanding as student learning 
result. 


Conclusions and Implications 


Assessment indicators identified in the national level tests of the two last years 
match against the three scientific literacy competencies, but to a different extent both in 
terms of balance, as well as cognitive depth. However, there are considerable changes 
between the Year 2018 and Year 2019 with regard to the assessment of scientific literacy 
competencies. For example, in the Year 2018 there are no items assessing student 
competence to explain phenomena scientifically, but in the Year 2019 there are 12 items 
assessing it. This leads to the inconsistency of the result interpretations by teachers and 
policy makers. 

Item assessment indicators reveal that the most substantial part of the assessment 
items assesses lower cognitive skills according to the SOLO. This implicates a huge 
disparity between the revised science curriculum and the assessment, as one of the main 
goals of the curriculum reform is to enable students to demonstrate the ability to solve 
problems and deal with complex situations. Thus, the research indicates the need for a 
better alignment between the learning content in the new curriculum and the assessment, 
in order to ensure successful reform implementation. 
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